Time-Lapse Light Field Photography With a 7 DoF Arm

A photograph taken by a conventional camera captures the average intensity of light at each pixel, discarding information about the angle from which that light approached. Light field cameras retain angular information about the rays they collect, allowing re-integration of rays during post processing. Consumer light field cameras have small apertures, and laboratory camera arrays with large baselines are expensive and not portable. In this paper we demonstrate time-lapse light field photography with the eye-in-hand camera of the Baxter robot. Using the eye-in-hand we can collect light densely and precisely over large distances. The collected rays can be refocused in software with varying lens and aperture properties to form conventional 2D photographs. This refocusing allows us to perform 3D reconstruction and segmentation and suggests approaches for enabling existing computer vision algorithms to robustly handle optically active surfaces. The techniques in this paper can contribute to robotic visual systems for object manipulation as well as media collection systems for virtual reality devices.